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The U.S. Navy conducted a study to determine the 
readability grade level of materials used in its technical training 
schools curricula as a basis for (1) establishing minimal reading 
competencies for those schools, (2) identifying problem textual 
materials used in those schools, and (3) choosing additional school 
curricula for readability analysis. Selection of schools used in the 
study was based ori annual completion, percent of students set 
and withdrawal for academic reasons. The readability analyses 
obtainpd^using the Computer Readability Editing System (CRES). 
calculated with the Flesch-Kincaid readability formula. The results 
indicated that the electronics technician curriculum had the most 
difficult readability grade level, 12.6. The least difficult course, 
machinery repairman, was found, to have a readability grade level of 
8.1. The elejptronics techni&ian curriculum also had the highest 
percentage of setbacks and the third highest percentage of academic 
attrition of the 11 courses studied. The fact that the mean 
readability grade level of the 11 sampled courses correlated 
positively with setback xates indicated that readability may 
contributing to the setbacK problem. Based on these results, 
recommended that readability grade level standards be set to 
the mean reading ability of the intended reader and that the 
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SECTION I 
INTRODUCTION 



Many sailors entering "A" schools lack the reading skills to cope success- 
fully with school reading materials. For example, at the Operations Special- 
ist "A" School, Dam Neck, Virginia, more fh^an 1,000 students per year require 
remedial reading instruction.! In addition, the Chief of Naval Operations 
has stated that a "substantial proportion of recruits read below the 10th 
grade leve\. Academic Remedial Training programs at the Recruit Training 
Commands and the Job Oriented Basic Skills program are limited 'in scope'and 
cannot overcome the entire Navy (reading) problem which has its roots in our 
public educational system. This situation is projected to become more criti- 
cal as we face the declining manpower pool of the 1980' s. "2 

According' to the most recent Navy figures, 25 percent of entering Navy 
recruits read at or below the ninth grade level. 3 An initiative addressing 
the need to i.mprove reading skills of unlisted personnel is contained in 
OPNAV Instruction 1510.11.4 The instruction establishes a policy of pro- 
viding "appropriate fundamental skills training interventions to upgrade 
basic competencies^ in support of military operations." The instruction 
further states that the Chief of Naval Education and Training (CNET) has the 
responsibility to: (1) design and implement training programs to achieve 
competency goals identified for Navy training courses and (2) determine the 
skills needed to comprehend technical, information. Because of the long-term 
involvement of the TAEG with the basic skills program of the NAVEDTRACOM 
\noi\' "^^""^^ ^"'"'"y' 1979; Aagard, Pereyra, and Kincaid, 1981; Brown, 
1982), the CNET tasked the TAEG to determine the readability grade levels of 
selected "A" school course reading materials. 6, 7 as part of the tasking for 
this study, TAEG completed an assessment of the readability level of essen- 
t^ial job reading material for nonrated Navy personnel (Hamel, Aagard, and 
Kincaid, 1982). - ' y . ' 

PURPOSE 

The purpose of the present study was to determine the readability grade 
levels (RGL) of materials used in the curricula of selected "A" schools (those 
with high throughput and/or attrition) as a basis for: (1) establishing 
minimal reading competencies for those schools, (2) identifying problem tex- 
tual materials used in these schools, and (3) choosing additional school 
curricula for readability analysis. 

InAVSWC 031507: Aug 1982 . ' • ' ' 

^CNOltr 204564 of 21 Apr 1980 

3CMI Recrurt Population Analysis Report produced by the Management 

Information and Instructional Systems Activity (MIISA) 
^OPNAVINST 1510.11, Enlisted Fundamental. Skills Training 

.SB^ic competencies include not only reading but'also mathematics and 
communications skills 
6CNET Itr of 13 July 1981 

^Mvanced First Term Avionics, a post-"A" School, was also included- in the 
study . \ ■ 

3 . 
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ORGANIZATION OF THE REPORT 

In addition to this introduction, the report contains three additional 
sections and two' appendices. Section II presents^ the approach ,used in the 
readability grade level analysis and the rationale for selecting the 
particular schools. Section III contains the study results. Section IV 
presents the conclusions and recommendations. Appendix A is a list of the 
sampled reading materials, with readability grade levels. Appendix B 
contains an example for one school of technical words, identified by the 
computer readability analysis not ordinarily in the vocabulary of beginning 
students. Such lists should prove useful for developing vocabulary 
instructional materials for students entering Navy technical schools. 
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' SEalON II 
APPROACH 

u- u^5^L^^^^i°" describes the approach taken in: (1) selecting courses in 
which difficult reading material might be a problem and (2r conducting the 
readability analysis. 

SELECTION OF COURSES - 

■ . Resources permitted the study of only a limited number of school cur- 
ricula^o a technique for prioritizing schools most likely to benefit from a 
readability analysis was required. The priorities established in selecting 
schools for this study were based on three criteria: (.t) annual throughput. 
(2) percent of students set back, and (3) percent of students attriting for 
academic reasons. ' 

• Table 1 shows the schools selected and data for the three selection 
criteria for the year ending 31 August 1982. Annual throughput- varied . 
considerably ranging from 3.247 for Avionics "A" School to 78 for Mineman 
a bchool. A 1 selected schools experienced a high rate of setback 
(ranging from 16 percent to 88 percent) and most experienced a high rate of 
academic attrition (8 out of 11 had 10 percent or greater). 

READABILITY ANALYSIS 

Instructional personnel of the 'schools (either an education specialist 
officer in charge) selected course material for .analysis which they 
judged to be most important in the curriculum. TAE6 was also furnished a 
curricuum- guide to determine the appropriate pages for analysis. These 

??pJhIm ?fri?"L^''M"J7SSl^'^/"°''^''"9'^° DOD specification dealing with 
readability (MIL-M-38784A. .Amendment 6, Department "of Defense. 1982) For 

tlZ ^nif^^'S^^K^^' & ^"'^ locatecj at NAS.Meraphis) ^TiateriaTs 

were col ected by a TAE6 representative. At the other schools, materials 
were sent after one or more telephone conversations. In al I cases TAE6 
personnel contacted school personnel knowledgeable about the school's 
curriculum to yerify that the selected materials were the most important 
ones for thei,r course, 'i • ■ 

v.o,H,K!i"f~^^I^'''^' Amendment 6, prescribes a technique f6r determining the 
readability of course materials and a technique for sampling materials for 
the ana ysis.. ,For a document containing 100 or -more pages of textual 
material. 10 samples of approximately 200 wcfrds are selected and a reada- 
?i! n^^f ^r- It""^^ obtained using the Flesch-Kincaid readability formula 
(Kinca d Fishburne. Rogers, and Chissom. 1975). This formula has two fac- 
tors: (1) sentence length in words and (2) word length in syllables It 
provides grade level according to the formula: 

Grade level = 0.39 (Average No. Words/Sentence) i- 11.8 (Average No 
Syllables/Word) - 15.59. 



ERIC 



S 



TABLE 1. 



Technical Memorandum. 83-2 



ANNUAL THROUGHPUT, PERCENT SETBACK, AND PERCENT 
ACADEMIC ATTRITION FOR SELECTED SCHOOLS* 



Course Title 



Annual 
Throughput 



\ 



Setback %■ 



Attrition % 



Avionics (AV) 3,247 

Electronics Technician (ET) 2,275 

Aviation Electrician's 

Mate (AE) " " . 1,572 

Advanced First Term 

Avionics (AFTA) 751 

Air Traffic Controller (AC) 745 

Data Systems Technician (DS) 535 

Aviation Antisubmarine 558 
Warfare Operator (AW) 

Machinery Repairman (MR) 391 

Strategic Weapons Systems 

Electronics (SWSE) 357 

Aerographer 's Mate (AG) 226 

Mineman (MN).. 78 



16 
88 

34 

39 

33 
18 
37 

25 

57 
35 
27 



11 

25 



4 

38 
8 

10 

11 

32 
16 
11 



*For year ending 31 August 1982. Data provided by CNET, Code N-2. i 

The obtained readability grade level corresponds to the grade level of 
reading ability required to understand the text. For example, a person 
reading at the 10th grade level should have full comprehension of a document 
written at the 10th grBde' readability level. 

The readability analyses were obtained .using the Computer Readability 
Editing System (CRES) described in TAEG Report 83 (Kincaid, Aagard, and 
• O'Hara, 1980). TAEG has used the CRES for readability analysii' of the 
Surf ace' Warfare Officers Scliool curriculum (Aagard, et al., 1981) and for 
essential reading material for enlisted personnel (Hamel, et al., 1982). As 
•in these studies, samples of text were keyed into the computer (or read in 
if available in machine-readable form), and the readability grade levels 
wore automatically obtained. The CRES also flags words not in a core 
vocabulary representing words a Navy trainee should know. These flagged 
words can be useful in identifying technical words (see appendix B) for 
glossaries for training students entering the particular '*A" school. 
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SECTION III 
RESULTS 



Table 2 summarizes the average readability gradfr levels, setback rates ' 
and attrition rates foV the, courses analyzed in this study. The data Te 
presented in descending order of their mean readability grade leve-1 The 
Electronics Technician curriculum waj. found to have the most difficult read- 
.^n f Ji? t.^'"'^' "^'""^ first-year college level. The second 

I J'k" Advanced First Term Avionir^, was found to have a 

least d?^??n i\^ T^' ^t'^l'^ ^h^' ^'^^ are- electronics courses. The 
orlll ?pvl ni ^'''u"''^ Repairman, was found to have a readability 

overal .uL Jho "lean readabi lity grade level is an important 

overall measure, the upper range of readability grade levels should be 
stressed when considering 1:he trainee's ability to comprehend material. For 
instance, one very difficult publication in a school ' s curriculum might be° 

aHlifv°nr H^wfr"/?;; '"'^^'"S ability matches the mean read- 

ability grade level of the course material. 

.r.w^HIff ""I^K °! academic difficulty shown in table 2 include percentages of 
acatlemic setback and academic attrition. It is interesting to note that the 
crvurse with the highest RGL. Electronics Technician, also had the highest^ 
Hnn'nJ'?h.°(i''^'^''^' ^^^^^ ^^'"'"^ ^'^^''^ percentage of academic at?ri- 
to a«;.Lf?hJ .J^pn'fh-^?^ ^^"^y- Correlations were computed 

hL?^ H L if °^ relationship between RGL and measures of aca- 

demic difficulty The correlation between mean readability grade level and 

rnrrlT.t^' '^.1^^'^^' ' = '^^'^ i s .cons i dered a moderate y h gh 
correlation although statistically not significant.8 The correlation 

0 nf "tole'^'^'S^ ^'?h' r^^'^ag^'of academic 'aUrUirwas 

bet":een%h1 two" variables? ^"'^"'^^ '''''' relationship^^ 

. Table 3 shows rec;ent data' (FY 82) foe the reading ability of Navy 
recruits. Al recruits were tested using level -D of the Gates-McGinitie 
Reading Tes (MacGinitxe; 1978). Thirty-two percent of recru ts we?e hown 
to have college level reading (above a grade level of 12.0). Recruits with 
this reading proficiency should not experience difficulty with" the readina 
materials encountered in any of the schools surveyed in this study. Forty- 

aJh Ld f?fh n^LlT"'?' "^t ''^^^^'.^^ ^""'^ "^^^^'"9 abilities between the 
8th and 12th grade levels. The reading ability of this group would permit 
them to comprehend only part of the Veading fnaterials surveyed in 'this study 
ara5p^p!lf"VS^ '^'''''^ '^'^'^^ abilities at or below the eighth ^ 

a^j'of^t^r^choSirsJrSe":;: ''''''''''' -^P-^-^ing te^t in 



^Probability of -obtaining a correlation of .49 (n=ll) .by chance is 19 
percent. Conventionally, -a probability of 5 percen is considered 
statistically significant. . ■ ^ ^^"^f^fireo 
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TABLE 2. MEAN READABILITY GRADE LEVELS (R6L), SETBACK RATES, AND ACADEMIC 
ATTRITION PERCENTAGES FOR SELECTED SCHOOLS 



Course Title 


Mean RGL 


RGL Range 


Setback %* 


Attrition 51^* 


Electronics Technician (ET) 


12.5 


n. 9-13.0 


88 


25 


Adv.anced First Term Avionics 

(AFTA) 


11.9 


11.7-12.0 


39 


4 


Mineman (MN) 


11.3 


9.1-14.4 


27 


11 


Avionics (AV) 


11.0 


10.8-11.3 


16 


11 


Air Traffic Controller (AC) 


10.9 


6.6-15.9 


33 


38 


Aviation Antisubmarine 
c Warfare Operator (AW) 


10.5 


9.4-12.1 


37 


10 


Aviation Electronics Mate 

(AE) 


10.3 


6.4-12.4 


34 


7 


rterograpner s nate ^mu; 


in n 


C 7 1 1 O 
D. /-1 1 .O 


OCT 
00 


1 c 
10 


Data Systems Technician (DS) 


10.0 


9.2-10.8 


18 


8 


Strategic Weapons Systems 
E-lectronics (SWSE) 


9.9 


9.2-10.3 


57 


32 


Machinery Repairman (MR) 


, 8.1 


7.1- 9.3 


25 


11 


Overall Mean 


10.5 









*For year ending 31 August 1982. Data provided by CNET, Code N-2. 
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TABLE 3. READING ABILITY* OF NAVAL RECRUITS (FY 82) 
^ i _ 



Grade Level Range 



Above 12.0 
10.1-12.0 
8.1-10.0 
5.1-8.0 
4.1-5.0 
Below 4.0 



Median = 10.9 79^054 
Mean = 9.8 



Number 


Percentag 


25,535 


32.3 


22,986' 


29.1 


14,707 


18.6 


12,895 


16.5 


2,539 


3.2 


390 


.5 



♦Reading scores were obtained from the CMI Recruit Population 
Ana ys,s Report produced by the Management Informat?Sn and 
Instructional Systems Activity (MIISA). '"a«.iun ana 
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SECTION IV 
CONCLUSIONS AND RECOMMENDATIONS , 

CONCLUSIONS 

The ov.erall^mean readability grade level of the school curricula sampled 
was 10.6. Comparing this_ to FY 82 figures on the average reading ability 
level of Navy recruits (X-= 9:8, ^median = 10.9, see table 3) there would 
appear to be a match between readability grade --fevel and reading grade level 
for courses sampled. However, difficulty range of the materials sampled 
varied widely within courses. Certain curriculum materials had readability 
grade levels considerably. higher than the reading ability of the intended 
users. 

The fact that mean reidability grade level of the 11 sampled courses 
correlated positively with setback rates indicates that readability may be 
contributing to the setback problem. This appears to be particularly true 
of those schools which embody Righ technology (e.g.. Electronics Technician 
f "A" School and Advanced First Term Avionics School). 

An examination of the curriculum materials used in the highly technical 
^ schools revealed the frequent use of many difficult technical words, required 
by the subject matter. The CRES analysis which calculates readablity grade 
level also flags uncommon words. Many of these uncommon words are technical 
words (e.g., gyrocompass) which have no substitute. Therefore, a supplemental 
list must be constructed for use with specialized materials. Appendix B 
presents an example of such a list for the Strategic Weapons Systems Elec- 
tronics "A" School. A list like this is a tool for producing glossaries and 
vocabulary exercises for Navy technical school students needing reading 
instruction specific to the course. The production of such materials is 
likely to be an important part of carrying out the requirements contained in 
OPNAVINST 1510.11. 

It should be noted that readability is just one measure of the compv^e- 
hensibility of textual material. Other factors of comprehensibility such as 
content, vocabulary, sentence structure, and format are not addressed by 
readability formulas. Nevertheless, a very high RGL, such as 16 or above, 
is indicative of a problem, and thus signifies a need for simplifying the 
material. The CRES is one aid for improving not just readability but compre- 
hensibility (e.^g., vocabulary, ^sentence construction, test item format). 

RECOMMENDATION^ 

1. In accordance with OPNAVINST 1510.11 which gives CNET the responsi- 
bility to set readability and/or comprehensibility standards, the following 
are recommended: 
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• set readability grade level standards to match the mean 
reading ability of the intended reader when curriculum 
mater iaj is being written or revised 

• use Gates-MacGinitie reading test results as one measure to 
derive target readability grade level requirements for 
particular schools 

• use the CRES or the manual method prescribed by DOD specifi- 
cation MIL-M-38784A, Amendment 6, to assess readability grade 
level of curriculum material 

• use the CRES to improve the comprehens ibi 1 ity of Navy curric- 
ulum material . 

2. Develop glossaries, wherever appropriate, particularly for highly 
technical curriculum materials. Although most Navy schools now have glossa- 
ries, increased emphasis should be placed on their development and use. The 
CRES is a good tool for developing glossaries and should be used where 
available for this purpose. 

3. Select additional courses for readability analysis based on the 
strategy for prioritization described in this study. Analyze the curricula 
of these courses as resources permit. In accordance with this recommenda- 
tion, the CNET should task an appropriate organization for a programmatic 
effort. It is anticipated that several man-years would. bp required for 
analysis of remaining "A" schools. 
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APPENDIX A 

READABILITY GRADE LEVEL OF SAMPLED MATERIALS 

The following are readability grade levels for each document analyzed 
for the school cited in section II of this report. 



AIR TRAFFIC CONTROLLERS "A" SCHOOL 



Title 



Charts and Publications 

USAF/USN NOTAMS 

Time and Basic Navigation 

Search & Rescue 

Air Traffic Control 
Procedures 

Airport Facilities and 
Lighting 

ASR Approaches - Part I 
ASR /^proaches - Part II 
ATC Radar- - Part I (ASR) 
Basic Radar Theory 
Avicition Weather 
NavAids 

Base Operations Laboratory 
Terminal Facility Equipment 
Control Tower Operator 

Overall 



Numb er 

Information Sheet 9.9.11 
CNTT-M1515 

Information Sheet 9.4.11 

Information Sheet 9.5.11 
CNTT-M1483 

Information Sheet 9.6. H 
CNTT-M1485 

Information Sheqt 
OPNAV 3710.7 Excerpts 

Information Sheet 9.12.11 



CNTT-M1139 

CNTT-1135 

CNTT-M1071 

CNTT-G87 

CNTT-MlllO 

CNTT-1338 



CNTT-M1516 
CNTT-M1239 



RGL 
15.9 

14.2 
6.6 

9.9 

12.1 

11.0 

13.1 
8.9 
11.0 
8.5 
10.1 
11.1 
9.3 
10.1 
11.7 



10.9 
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Title 



AVIATION ELECTRONICS MATE "A" SCHOOL 

Number 



Positive and Negative Numbers 

Fractions (Basic Math) 

Powers of Ten 

Magnetism 

Magnetic Theory 

Aircraft Wiring Practices and 
Basic Electrical Troubleshooting 

Basic Electronic Circuits 

Aviation Electrician ' s Mate ■ 

Overall 



CNATT-M25 

CNATT-P-4968 

CNTT-M707 

CNATT-M544 

CNATT-M154 

CNTT-M1012 

CNTT-M971 
NAVEDTRA 10348-D 



RGL 
''•8.1- 

6.4 
11.0 
10.3 
11.2 

9.5 

10.5 
12.2 



10.3 



ADVANCED FIRST TERM AVIONICS COURSE (AFTA) SCHOOL 

Title Number 

Trainee Workbook, Phase IV, Unit 1 CNTT-M491 

Trainee Workbook, Phase III, Unit 2 CNTT-M488 

Trainee Workbook, Phase III, Unit 1 CNTT-M542 

Trainee Workbook, Unit 2 CNTT-M1453 

Overall 11.9 



RGL 
11.8 
11.7 
11.5 
•12.0 



Title 



AEROGRAPHER'S MATE "A" SCHOOL 

Number 



Decoding & Plotting of the 
International Analysis Code 

Decoding RadFo Messages and 
Plotting Radiological Fallout 
Diagrams 

The APT Predict Message and 
Tracking Board 



CNTT-L186 

3ABR25130-2-PT-305J 
3ABR25130-2-PT-407A 



RGL 
8.1 

7.2 
5.7 



ERIC 
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AEROGRAPHER'S MATE "A" .srHnOL (continued) 



Title 



Skew T, Log P Diagram 

Pilot Reports (PiRep Code) 

Cloud Forms 

Types of Observations 

Oceanic Circulation 

Sound Ray Theory 

Basic Principles of Sea 
and Swell 

Meteorological Satellite 
Terms and Equipment 

Properties of Sea Water 

Aerographer's Mate 3 & 2 

Weather /or Aircrews 

Federal Meteorological Handbook 

Surface Synoptic Codes 

Overall 



Number 
3ABR25130-2-PT-408A 
3ABR25130-SG-112 
3ABR25130-WB-104B 
3ABR25130-WB-110 
CNATT-L129 
CNATT-L131 
CNTT-L149 

3ABR25130-2-PT-308A 

3ABR25130-2-PT-401 
NAVEDTRA 10363-E 
AFM 51-12 
FMH-IB 
FMH-2 



10.0 



Title 



Introduction to AM Communications 
Unit 1, Module 1, Volume III 

Introduction to AM Communications 
Unit 1, Module 1, Volume II 

Overall 



AVIONICS fAO.AT,AX) "A" SCHOOI 
Number 
CNTT-M1311 



CNTT-M1314 



11.0 



T 
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AVIATION ANTISUBMARINE WARFARE OPERATOR "A" SCHOOL 



Title Number RGL 

AW(Al) Prerequisite Mathematics CNTT-M1178 9.4 
Students' Guide, Vol. 1, 

Phases I & II CNTT-M1184 12.1 

Overall 10.6 

DATA SYSTEMS TECHNICIAN "A" SCHOOL 

Title ^ ' Number RGL 

Trainee's Guide for the 10.2 

COMTRAN TEN 

Learner's Guide, Vol. I PX-10773-1 9.7 

Learner's Guide, Vol. II PX-10773-2 10.8 

Trainee's Guide, Phase A-1 9.2 

Overall 10.0 

ELECTRONICS TECHNICIAN "A" SCHOOL 

Title Number RGL 

Telecommunication Systems ET/A-18691 13.1 

Radar (AN/SPS-IO) ET/A-18093 11.9 

Advanced Electronics and ET/A-18465 13.0 
Circuit Analysis 

Overall 12.6 

MINEMAN "A" SCHOOL 

Title Number RGL 

Student Guide, Unit 2 10.5 

Student Guide, Unit 3 9.4 

Student Guide, Unit 5 11.2 
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' MINEMAN "A" SCHOOL (continued) 

Title * ^ Number 

-J! 

Student Guide, Unit 7 

Student Guide, Unit 8 

Technical Manual NAVSEA OP 3504, Vol. 1 

-Technical Manual . -'NAVSEA OP 4410, Vol. 5 

Technical Manual ■ NAVSEA OP 44-10, Vol. 1* 

Technical Manual •' NAVSEA OP 2572 

Technical Manual - NAVSEA OP 3529 

Overall ,' ^ . 11.3 

MACHINERY REPAIRMAN "A" SCHOOL 

Title . . Number 

Student's Guide Volume 1, Phase 1 

Student's Guide Volume 1, Phase 2 • 

Student's Guide Volume 1, Phase 3 

Student's Guide Volume 1, Phase 4 

Student's Guide Volume 1, Phase 5 ^ 

Student's Guide Volume 1, Phase 6 _ --' --- 

Machinery Repairman 3 & 2 NAVEDTRA 10530-D 

Overall 8.1 
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STRATEGIC WEAPONS SYSTEMS ELECTRONICS "A" SCHOOL 
Title Number 



Basic Digital Computer 
Fundamentals 

Inductance 

Trainee Guide 



Overall 

0 



NAVTECHTRA 121-0142 

Rev, A, SSWB-BDCl, Vol. 2 

002/2-3-4 

NAVTECHTRA 121-0142 
Rev. A, Vol. 1 

9.9 
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APPENDIX B' 

TECHNICAL WORDS IDENTIFIED BY CRES ANALYSIS 



STRATEGIC WEAPONS SYSTEMS ELECTRONIC "A" SCHnni 



bit-by-bit 
CEMF 

coefficient 

(coun tere 1 ec tromot i ve 

capacitive 

cathode-ray 

CCW 

CREO 

CW 

component 

crystal-controlled 

comparator 

converter 

digital 
DMM 

DSflux 

five-bit 

flip-flop 

four-bit 

free-running 

fractional 

half-cycle 
HZ 

inductive 
inductor 

indue tosyn 

inertia 

KHZ 

Kirchhoff 
K-F/F 
lenz 
LSD 

microheneries 

microsecond 

misaligned 

multivibrator 

mylar 

NMSD 

0D5 

OPS 

oscilloscope 

over-r^nge 

permeability 

pin- jacks 

potentiometer 

probe 



repulsion 
resolver 
representation 
repunched 

51- S 

52- S 

self-inductance 

self-synchronous 

synchros 

sine-wave 

sign-number 

SSWB 

tp"A" 
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CNAVRES (02) 
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